DOCUMENT RESUME 



ED 376 615 



EA 026 340 



AUTHOR 
TITLE 



INSTITUTION 

SPONS AGENCY 
PUB DATE 
NOTE 
PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Patrick, Edward M. ; Dawson, Judith A. 

A Description and Analysis of Five Teacher 

Supervision/Evaluation Systems: Summary of Case 

Studies , Findings , and Recommendations . 

Research for Better Schools, Inc., Philadelphia, 

Pa. 

Pennsylvania State Dept. of Education, Harrisburg. 

Oct 85 

57p. 

Reports - Research/Technical (143) 
MF01/PC03 Plus Postage. 

Elementary Secondary Education; ^Faculty Development; 
Program Implementation; School Districts; Staff 
Development; *State Departments of Education; Teacher 
Effectiveness; *Teacher Evaluation; *Teacher 
Supervi s i on 
^Pennsylvania 



ABSTRACT 

During the 1984-85 academic year, the Pennsylvania 
Department of Education (PDE) initiated efforts to reform school 
districts' teacher supervision/evaluation (TS/E) procedures. The goal 
was to improve the quality of classroom instruction and develop more 
consistent, meaningful teacher evaluations. This paper describes a 
study conducted by Research for Better Schools that examined the 
teacher-evaluation systems in five Pennsylvania school districts. 
Data were obtained through: (1) interviews with a total of 20 central 
office staff, 37 administrators and/or trainers, and 194 teachers; 
(2) analysis of district records; and (3) followup contacts with a 
sample of district officials. The following' critical elements in 
designing and implementing new TS/E systems were 

identified — introduction and generation of staff support, system 
design, training, and implementation/continuation. Key issues include 
the relationship between supervision and evaluation, the selection of 
trainers, and system monitoring. Alternative modes of 
supervisi on/evaluat ion and centers for teacher training are two 
promising innovative practices. Findings indicate a definite and 
continuing support role for the PDE in districts' TS/E improvement 
efforts. Several recommendations are made for improving PDE's role. 
(LMI) 
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Introduction 



In the 1984-85 school year, the Pennsylvania Department of Education 
(PDE), at the governor's behest, began to actively encourage Pennsylvania 
school districts to reform their teacher supervision/evaluation (TS/E) 
procedures. Its goal was to improve the quality of classroom instruction 
and make teacher evaluation more consistent and meaningful. As an initial 
step, PDE in cooperation with the state Intermediate Units, sponsored a 
series of 2.9 regional two-day workshops to familiarize school staff with 
the components of an effective TS/E system and staff skills required to 
implement them (McGreal, 1983). In 1985-86, PDE plans to provide districts 
with more specific models and how-to-do-it information based on existing 
TS/E systems in the state. This will be followed by technical assistance 
in the development and implementation of similar TS/E systems. 

To obtain some of the data necessary to this effort, PDE commissioned 
RBS to design and conduct this study of five school district TS/E systems. 
The five districts were selected by PDE after a questionnaire and phone 
survey determined which districts in the state had TS/E systems either in 
place or evolving. An attempt was also made to include urban, suburban, 
and rural sites, among the districts selected. In the end, Abington 
Heights, Pittsburgh and Upper Perkiomen were selected because they had 
Madeline Hunter-type instructional improvement/supervisory models in use. 
Suburban East Penn and rural Tamaque were selected for their evolving 
alternative TS/E systems. 
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Method 

The study took a case study approach. Separate interview protocols 
for district central office staff, administrators/trainers and teachers 
were developed and pilot tested. These were based in part on the protocols 
used in a recent Rand Corporation study of effective teacher evaluation 
practices (Wise, et al., 1984, personal communication). The interviews 
also addressed some 25 how-to-do-it type questions posed over the past year 
by Pennsylvania school officials in TS/E related meetings and conversations. 

In addition, the terms "supervision" and "evaluation" were defined at 
the beginning of each interview to ensure a common understanding among 
participants and researchers. Supervision was defined as that cycle of 
activities between a teacher and an administrator or supervisor that is 
intended to improve the teacher's ability to perform in the class. It is 
primarily improvement oriented and can focus on instructional techniques, 
class management, planning, implementation of the distruct curriculum, etc. 
It can focus on teachrr improvement goals and/or district goals. Evalua- 
tion was defined as the culmination of the supervision cycle wherein the 
administraor or supervisor makes a summary judgement or evaluation of the 
teacher's classroom performance for personnel or accountability purposes— 

usually on an annual basis. 

A three-phase study procedure was devised. First, descriptive TS/E 
system background and policy information (e.g., policy manuals, training 



*In Pennsylvania, the annual evaluation of teachers and other 
professional employees is mandated (c.f. Pennsylvania Education code, 
Chapter 351, 351.21 Rating Form). The state form for this purpose, the 
Temporary Professional Employee/Professional Employee Rating Form, is 
referred to here as DEBE-333. 
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materials, observation instruments, budget data) was solicited from each of 
the five districts. A brief pre site-visit background information ques- 
tionnaire was also used. Second, in the spring of 1985 two researchers 
spent three days on site at each district, (six person days per site) 
interviewing school staff. At each site they interviewed central office 
staff and district trainers for approximately three hours, administrators 
for one-and-one-half hours, and teachers (separately or in small groups) 
for a half -hour. In the four smaller districts, Tamaque, Upper Perkiomen, 
Abington Heights, and East Penn, the number of administrators (including 
central office staff) and teachers interviewed ranged from 7 to 10 and 31 
to 43, respectively. In Pittsburgh, the largest district, 22 adminis- 
trative staff and 51 teachers were interviewed either separately or in 
small groups. Across the five districts, a total of 20 central office 
staff, 37 administrators and/or trainers, and 194 teachers were inter- 
viewed . 

Interviews with central office staff touched upon development 
processes, system designs and operations, and outcomes, strengths, and 
weaknesses. Interviews with administrators focused primarily on their 
specific roles in supervision and evaluation, the utility of the training 
they received, their perceptions of their impact, perceived system 
strengths and weaknesses, and recommended changes. Interviews with 
teachers focused primarily on their perceptions of the utility, impact and 
f aimers of the training/supervision and evaluation procedures, the speci- 
fics of the procedures, perceived strengths and weaknesses, and recommended 
changes. Several questions were asked of all staff to determine if they 
shared common perceptions of the goals, procedures and impact of the TS/E 
system. 
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Lastly, follow-up contacts with select district officials were made 
for further clarification of the information collected. In essence, 
district staff verified the accuracy of the program descriptions and 
implementation procedures cited in the report. 

Related Research 

Two recent publications address factors related to effective teacher 
supervision/evaluation systems. McGreal (1983) identified nine factors 
shared by effective teacher evaluation systems: (1) focus on instructional 
improvement as opposed to accountability; (2) correspondence between the 
major purpose(s) of supervision/evaluation and procedures and instrumenta- 
tion; (3) separation of evaluation of teaching (supervision for improvement 
purposes) from teacher evaluation; (4) use of some form of goal setting 
procedures (individualization of supervisory procedures); (5) a narrow 
focus on teaching and a common understanding among administrators and 
teachers of the teaching act; (6) use of a modified clinical supervision 
format; (7) use of alternative sources of data; (8) different requirements 
for tenured and non-tenured teachers; and (9) in-depth training or staff 
development for both administrators and teachers. 

The second publication was a recent Rand study (Wise, et al., 1984) of 
teacher evaluation systems in 32 school districts. Four of the districts 
were studied intensively. Researchers found well-developed supervision and 
evaluation systems in only a few of the districts. The problems most 
frequently cited in the districts were: (1) lack of sufficient resolve and 
competence on the part of principals to evaluate accurately; (2) teacher 
resistance to and lack of support for district evaluation programs because 
of apathy and perceptions of inconsistent criteria and subjective 



variability in the evaluation process; and (3) lack of credibility in cases 
where generalists (principals) evaluated specialists (high school and/or 
special subject area teachers) ♦ The most commonly cited benefits, even in 
less well-developed teacher evaluation systems, were: (1) improved adminis- 
trator-teacher communications; and (2) increased teacher awareness of 
instructional goals and classroom practices, due in part to the use of 
modified clinical supervision procedures. 

The characteristics which distinguished the more successful from the 
less successful districts were: (1) provision of top-level leadership and 
resources for the evaluation program; (2) provision of training for evalua- 
tors in the skills required to evaluate effectively; (3) teacher-evaluator 
goals and procedures; and (4) development and implementation of evaluation 
procedures and support systems that are integrated with the district's 
overall goals and organizational structure. 

The Report 

This report consists of brief summaries of the five case studies . 
It also presents a discussion of findings and issues critical to the 
effectiveness of TS/E systems, and includes recommendations for other dis- 
tricts considering the development or revision of TS/E systems and impli- 
cations for organizations planning assistance to these districts. A sepa- 
rate document contains a detailed case study of each of the five districts. 

Study Findings 

Analyses of the data generally indicate that the study results provide 
support for many of the characteristics of effective TS/E systems identified 
by the Rand study and McGreal's publication. The study also identified 
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over 20 specific process factors/issues critical to the initiation, design, 



start-np. implementation and maintenance of effective TS/E systems. 



Summary of Case Study 1; Ablngton Heights School District 



The Ablngton Heights School District includes seven schools, four 
elementary, two middle, and one high school. It employs 16 administrators 
and 223 teachers for its 3,364 students. The district uses a modified 
DEBE-333 evaluation procedure. In 1979, the district also instituted a 
Hunter-based instructional/supervisory improvement program (Program for 
More Effective Teaching-PMET) . 

TS/E Program History and Philosophy 

PMET involves instructional-model training for administrators and 
teachers, and clinical supervision training for administrators. It was 
initiated in the district by a confluence of events that occurred in 
1977-78, First, the superintendent developed an interest in Long Beach „ 
California 1 s application of the Hunter model; second, a teachers 1 associa- 
tion survey revealed discontent with current supervisor /evaluation proce- 
dures; and, third, initial district attempts to self-train staff in TS/E 
fell short. Subsequent visits to an exemplary TS/E District (Newport News, 
VA) by a special representative committee resulted in the district taking a 
number of actions, including: (1) hiring a consultant from Newport News; 

(2) selecting and training a district trainer; (3) pilot testing a similar 

program; (4) developing and refining the PMET program; and finally, (5) 

implementing the program. 

District administrators view PMET as a process to upgrade the quality 

of teaching and supervision. The MET philosophy is one of helping staff 

increase skills through training and supportive follow-up supervision. 

Administrators see PMET/supervision and teachers 1 DEBE-333 annual 

evaluation as related but not formally connected. 



ERIC 



ERIC 



Staff Development 

Most district administrators received at least ten days of training in 
Hunter's elements of effective instruction and clinical supervision proce- 
dures. From the second to the fourth year of the program, they also 
received periodic follow-up coaching in observing/conferring by the 
district trainer. For teachers, training consisted of five days of sessions 
on Hunter's model over a five- or six-week period. Between each session, 
time was provided for practice, and coaching by the district trainer. 
Training was followed by regular observations and reinforcement by 
administrators/supervisors. Four or five small groups of teachers were 
trained each year from 1979-80 through 1982-83. Elementary school staff 
were trained first and secondary school staff last. 

TS/E Organization 

The supervision and evaluation process is carried out by principals, 
assistant principals, and eight subject area coordinators/supervisors. 
Principals and their assistants generally observe in their own buildings 
while supervisors observe across buildings. 

The district requires two formal observations of tenured teachers, and 
four of untenured teachers each year. These observations may be announced 
or unannounced, and generally last a class period. Observations involve 
anecdotal note-taking, lesson diagnosis, post-conferencing and a formal 
write-up. The focus is largely on PMET instructional concepts as well as 
on other aspects of teaching as needed. All formal observation reports are 
shared with central office staff. Principals' annual evaluations of teachers 
are based on their observations and other interactions with teachers. 
Staff are expected to spend about 20 percent of their time on TS/E. 
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Monitoring/Evlauation 

The superintendent and his assistant monitor implementation of the 
TS/E system by periodically reviewing observation and evaluation reports, 
and by sitting in on one teacher observation/conference with each adminis- 
trator each year. System evaluation has also involved formative evaluation 
of training. 

Project Costs 

Project costs were approximately $40,000 the first year and $20,000 
per year thereafter. The major expense incurred was salaries for substitute 
teachers called in to replace staff taken out of class for training. 

Administrators * Perceptions 

All staff found their training in PMET and clinical supervision 
valuable and effective. Most also expressed a need or desire for more 
training either in additional supervision process skills and/or other 
content areas related to teaching. Administrators, in particular, indicated 
that PMET helped teachers improve their instruction and increased their 
awareness of effective teaching. They cited the common language of instruc- 
tion and the common observation-expectations of what comprises good teaching 
as the major strengths of the system. The annual DEBE-333 teacher evalua- 
tion, on the other hand, was viewed as somewhat of a pro forma exercise 
that had little impact on teacher improvement. The one problem that 
administrators reported was the need to periodically renew attention to the 
implementation of TS/E to prevent it from reverting to a pro forma process. 
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Teachers * Perceptions 

Almost all teachers commented favorably on the usefulness of their 
PMET training. One-third cited specific examples of how they had changed 
their teaching practices as a result. Twenty-five percent reported that 
administrators' observations were helpful, while close to 60 percent 
attributed 20 percent cr more of their teaching expertise to participation 
in the PMET program. Teachers generally agreed with administrators that the 
DEBE-333 evaluation was a pro forma exercise. 

Teachers cited several program strengths. They said that it: helped 
them to systematize and organize instruction; provided a common language of 
teaching; increased communications between teachers and principals; and 
clarified TS/E expectatioha. The most commonly cited problem, mentioned by 
about 25 percent of the teachers was that the PMET program was misrepre- 
sented. It was sold as a program to help them but later was used to eval- 
uate them. At least a third of the teachers also felt that the TS/E system 
had no teeth to it. Above average teachers were not recognized and marginal 
teachers continued to exist. 

Central Office Staff Perceptions 

Central office staff cited the common language of instruction, 
increased administrator supervisory skills, increased communication and 
trust, positive reinforcement and help for teaches, and the modified 
DEBE-333 as prevailing program strengths. Major problems perceived by 
central office staff included traces of continued staff resistance to the 
program at the secondary level, the need for more systematic follow-up for 
teachers requiring assistance, the need to formally relate DEBE-333 to 
PMET, and the need to refocus attention on TS/E. 

,13 
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On the basis of this experience, the major advice offered to other 
districts contemplating a similar program was to involve all levels of 
teachers from the beginning in program planning and conceptualization, 
thoroughly train a district trainer, plan for at least a five-year commit- 
ment, and develop the program alsowly to build staff trust. 

Overall Program Significance/Implications 

The Abington Heights School District program was one of the first 
programs in the state to address instructional improvement through a 
systematic staff development/supervisory program. With a program that 
incorporates many of the features recommended by McGreal (1983), the 
district achieved many of its goals in this area. Several implications can 
be drawn from this school district's experience. Among them are: (1) the 
importance of using a represetative group process; (2) engaging in site 
visits of exemplary programs; (3) conducting a pilot test during the 
program selection phase of a new instructional supervisory program effort; 
(4) the importance of clearly defining the relationship between instruc- 
tional supervisory improvement efforts and a district's annual teacher 
evaluation procedure; and (5) paying attention to continued long-term 
follow-up, including emphasis on district resources for the program so that 
it continues to be perceived as a priority. 
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Summary of Case Study 2: East Penn School District 



The East Penn School District consists of 12 schools, nine elementary, 
two junior highs, and one high school. It employs 20 administrators, 300 
teachers and 5,800 students. The district instituted a modified DEBE-333 
evaluation procedure in 1981 and trained administrators in Dick Manatt's 
supervisory model. The district also recently initiated Project ProCEED 
(Professionals Committed to Excellence in Education) to broaden teachers 1 
instructional skills. 

TS/E Program History and Philosophy 

Discontent with the quality and consistency of teacher evaluation led 
to the formation of a Teacher Evaluation Committee in 1980-81. This 
committee modified the DEBE-333 and produced Tips for Teachers , a compendia 
of instructional guidelines for school staff. Administrators were also 
trained in Manatt's model of teacher supervision in the summers of 1981, 
1982, and 1983. Continued district interest in teacher staff development 
resulted in Project ProCEED, part of the district's 1982-83 long-range 
plan. Project ProCEED consists of six district developed and taught 
instructional modules. The modules address brain development /learning 
styles, teacher expectations and student achievement, Bloom's taxonomy, 
students with special needs, time and classroom management, and student 
motivation. Staff instruction in Project ProCEED content commenced in 
February 1984. 

In order to coordinate and integrate the district's evolving TS/E and 
staff development activities, the new district superintendent (October 
1983) hired an external consultant, Harvey Silver, to work with administra- 
tive and Project ProCEED staff in the summer of 1984 and during the 1984-85 
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school year. District efforts to the above end are still underway. 
Closure is expected in the 1985-86 school year. The district views super- 
vision and evaluation as part of the same overall process. Supervision is 
viewed, however, as a process intended to help teachers improve, and to 
support their professional and personal development. Evaluation is the 
summative rating reached after a series of observations. 

Staff Development 

To date, approximately 65 teachers and 15 administrators have partici- 
pated in the six days of training involved in Project ProCEED. Administra- 
tors received several one-half days of training each summer (1981-1983) in 
Dick Manatfs model of teacher supervision. In the summer of 1984 they 
received five days of consultant-led training in both leadership and 
learning styles as well as exposure to Hunter's instructional model. In 
addition, several small groups of administrators each engaged in a day or 
two of TS/E-related problem solving with the consultant in the 1984-85 
school year. Teacher training involves one day of class instruction in 
each of the six instructional modules. Application-related homework 
assignments are required for each module. These are discussed in subse- 
quent training sessions 

TS/E Organization 

The district requires two observations of tenured and four of non- 
tenured teachers each year. Teachers are observed for the entire lesson or 
period. At least one observation per year must involve a structured 
pre-conference using guidelines established by the district. All observa- 
tions are followed by a post-conference. Observation reports are routinely 
shared with central office staff. Tenured teachers are rated once, and 
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non-tenured staff twice, per year. Ratings are based on the observation 
reports and administrators 1 judgments of teachers 1 total performance during 
the rating period • 

Tenured secondary staff are observed at least once each year by an 
administrator and once by a curriculum supervisor • Tenured elementary 
staff are also observed by their respective administrators • Under the 
current plans, elementary staff will also be observed by the curriculum 
supervisors. Individual teachers may be observed only once every two or 
three years by the curriculum supervisors due to their observation loads. 

Monitoring/Evaluation 

The superintendent monitors the implementation of the district's TS/E 
system by reviewing all of the formal observation and evaluation reports 
submitted by administrative staff. He provides feedback to individual 
administrators at regularly scheduled goal setting/progress report meetings 
three times per year. Project ProCEED has been evaluated through post- 
training questionnaires soliciting participants 1 reactions and suggestions. 

Project Costs 

The district's recent efforts to improve teachers' performance via 
Project ProCEED appear to be running about $30,000 per year. Half of these 
monies support a half-time project director. The other half covers salaries 
for substitutes used to replace teachers taken out of class for training. 
Consultant fees for administrator training were $5,400 over the past year. 

Administrators 1 Perceptions 

The most common program strength perceived by administrators was that, 
in comparison to past procedures, some standardization had been introduced 
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into the TS/E process. Conversely, administrative staff made a number of 
other comments that indicated that even more standardization was needed. 
That is, teachers need to be trained in a single model of instruction to 
bring their expectations and understandings in line with those of adminis- 
trators. 

Most administrators spoke positively of the training that they had 
received and indicated that it caused them to change their behaviors. All 
were receptive to more training. Several desired more expertise in 
conferencing and helping teachers to improve. 

Administrators shared mixed perceptions of the impact of supervision. 
About half felt that it did help teachers improve. Others perceived a 
variety of problems both in teachers' receptiveness to the process and in 
the clarity of the district's TS/E policies. Administrators also saw the 
district's evaluation process as an improvement over past procedures. No 
strong comments were made, however, regarding the degree to which it helped 
teachers improve. 



Teachers' Perceptions 

The program's major strength, according to teachers, is the common 
perception it has produced among staff that the district is supporting the 
professional development of teachers. Almost all teachers valued the 
Project ProCEED training that they had received. Many also perceived that 
the district was attempting to upgrade the TS/E system. They saw the 
current system as an improvement over past procedures. 

There ware mixed perceptions of the impact of teacher observations. 
The great majority of the teachers saw them as being pro forma in nature. 
Teachers newer to the district and those less experienced reported that the 
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observations had greater utility and influenced their teaching behaviors. 
To a large extent staff were indifferent to the end-of-year staff rating. 
The project ProCEED training, however, was viewed quite favorably. Close 
to half of those staff who experienced ProCEED attributed 20 to 25 percent 

of their current teaching skills to their participation in the program. 
The major problem in the program is that a third to one-half of the 

staff perceive that there is variability among and between schools and 

administrative staff, and from year-to-year, with regard to the focus and 

rationale of the observations. 

Central Office Staff Perceptions 

Central office staff perceived the program's major strengths as the 
training administrators had received, the help provided to some teachers, 
the positive reaction to and effects of Project ProCEED, and the general 
awareness on the part of all staff of the district's efforts to improve the 
TS/E system and instruction. 

They also cited several problems. Among them were continued incon- 
sistencies in implementation of the TS/E system, the failure to clearly 
communicate the district's expectations regarding effective teaching to 
teachers, the need to clarify the relationship between the district's 
training program for teachers (Project ProCEED) and teacher supervision and 
evaluation, and the need to train both administrators and teachers in a 
well defined model of instruction. 

As a result of their experience, central office staff would advise 
other districts planning a TS/E program to have a clear vision of what the 
TS/E program will look like at the end, communicate clear expectations to 
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all staff, involve all staff in planning, train all staff, and work toward 
commitment and not just simply compliance. 

Overall Program Significance/Implications 

East Penn has made definite strides in its attempt to upgrade adminis- 
trators' TS/E skills and set up a quality staff development program for 
teachers. The district's Project ProCEED is an example of the planning, 
resource allocation, and commitment required to establish a quality pro- 
fessional improvement program. Several implications can be drawn from East 
Penn's efforts. First, all staff require training in a common instructional 
model if a TS/E program is to experience optimal success. Second, the 
relationship between new staff development programs and supervision programs 
needs to be clearly specified to all staff. Finally, staff require time to 
practice and absorb discrete parts of new instructional models. Coaching 
and feedback are required. Districts should avoid placing too much content 
before staff in too short a time. Otherwise, mixed signals regarding 
expected content applications and observation expectations may result. 
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Summary of Case Study 3: Pittsburgh Public Schools 



The Pittsburgh Public School District comprises 88 schools (53 elemen- 
tary, 17 middle [6-8], 15 high schools and 3 special education centers), 
253 administrators, 977 elementary school teachers, and 2,089 middle, 
secondary, and special school teachers- The district recently adopted a 
modified DEBE-333 teacher evaluation procedure which incorporates instruc- 
tional language and criteria derived f n *i the iistrict's Madeline Hunter- 
based, Pittsburgh Research-based Instruction il Supervisory Model (PRISM). 

TS/E Program History and Philosophy 

The PRISM program was initiated by the superintendent, who assumed 
office in September 1980. A needs-assessment study commissioned by the 
superintendent established staff evaluation as a key district priority. An 
instructional leadership committee subsequently recommended that a district 
staff development team (SDT) be formed to conceptualize and implement an 
improvement effort. That effort came to be known as PRISM. The five 
member staff development team, formed in June 1981, received in-depth 
training from national consultants, developed training materials and 
procedures, and began implementation of the PRISM program in September 
1981. 

PRISM is designed to make good teachers and administrators better 
while at the same time identifying those who need significant improvement. 
The program has three goals: (1) to foster effective teaching through 
staff development programs designed to upgrade the observation and evalua- 
tion skills of principals; (2) to establish a common language of instruc- 
tion and a set of standards for personnel evaluation; and (3) to provide 
clinical teaching experiences for both principals and teachers. 
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The intent is to place professionals in a helping relationship with 
one another to bring about positive improvements. If teachers and adminis- 
trators are unable to improve their performance after careful role clarifi- 
cation, reasonable observation and feedback, and specific training, then 
appropriate due process actions will be taken. A five-year timeline was 
proposed for the program with the intent that PRISM would become the 
standard for the formal evaluation of teacher performance once it was 
disseminated district-wide. 

Staff Development 

Administrator training commenced in September 1981. Principals and 
supervisors were the first to be trained in the district's incremental, 
aulti-year, staff development program. Most assistant principals and deans 
were trained in 1982-83. All received: (1) five days (30 hours) of PRISM 
I, Stage 1 training in Hunter's instructional theory and clinical supervi- 
sion model; and (2) five days of PRISM I, Stage 2 PTOC training (practice 
in lesson planning, teaching, observing and conferring). They also 
received 30 hours of supplemental staff development each summer since 1983 
in PRISM II. The goal of PRISM II is to enhance the instructional leader- 
ship knowledge, attitudes and skills of administrators (mini-workshop 
sessions addressing both district and individual improvement goals). In 
addition, they have: (1) received follow-up coaching at least four times 
per year since 1981 from SDT members who have co-observed and analyzed a 
lesson with them, observed them post-conference the teacher and provided 
them with feedback on their conference; and (2) participated in support 
networks which have met 3 or 4 times a year for one-half day since 1983-84 
to air TS/E implementation practices /problems. 
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Training teachers in PRISM I instructional theory by principals, 
vice-principals, and supervisors, with the back-up support of SDT members, 
began in the 1982-03 school year and has continued to the present. 
Inservice was accomplished through training activities taught and coordi- 
nated by administrators, and through principals' required twelve-a-month 
teacher observations and conferences. Training took place during the 
district's five, one-half day inservice sessions and, in some cases, during 
those teacher preparation periods allocated for instructional improvement. 
Hie total number of hours of training per year has varied in schools. The 
average has been approximately four hours of PRISM I each year over the 
past three or four years in addition to an average of four or five observa- 
tions/conferences per teacher per year. 

Pittsburgh initiated the Schenley High Teacher Center (SHTC) program, 
PRISM III, in the 1983-84 school year. The program's goal, in brief, was 
to provide an in-depth eight-week renewal program for all of the district's 
secondary teachers. To date, half of the district's secondary teachers 
have completed the program. During the eight weeks at the teacher center, 
the visiting teacher: (1) observes exemplary teaching in an ongoing school 
setting; (2) practices new instructional techniques and skills; 
(3) receives detailed feedback and analysis on that practice; (4) applies 
instructional theory to practice; (5) receives an update in the content of 
his/her subject area(s); (6) reviews and discusses recent research on 
effective teaching, classrooms, and schools; (7) studies adolescent 
behavior and its implications for effective teaching; and (8) researches a 
chosen topic for personal growth# The program also involves a follow-up 
support structure and liaison with high school principals to assist 
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teachers in applying what has been learned at the teacher center. The 
district plans to open a similar Elementary Teacher Center, PRISM IV, in 
1985-86. This would provide four weeks of in-depth clinical teaching 
experience to all of the district's elementary school staff. 

Requirements and Organization 

District administrative staff are responsible for carrying out the 
districts TS/E system. They are expected to spend approximately 20 
percent of their time on supervisory related activities. The observation 
load ranges from 19 to 25 teachers in elementary schools and from 100 to 
200 teachers in some secondary schools. Principals have the final 
responsibility for annual staff ratings. 

The district has required school administrators (principals, assistant 
principals and supervisors) to conduct 12 PRISM-related supervisory observa- 
tions (10-15 minutes) and conferences (10-15 minutes) per month (1982-1985). 
When the district shifted to longer n ref inement-type" observations and 
conferences, the requirement was reduced to six per month. The number of 
times teachers have been observed each year has ranged from 3 or 4 to 10 or 
12 depending on the size of a school's faculty and the number of assigned 
administrators. Records of supervisory observations have not been shared 
with central office staff. In the first year or two of the program, 
observations focused primarily on providing positive support and reinforce- 
ment for what teachers were doing well. As the program matured, they 
broadened to also address areas that impeded lesson progress. 

The district also requires that tenured teachers be observed/rated 
once per year and non-tenured staff twice per year. Observations for 
rating purposes are announced as such and employ rating observation forms 
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and report formats that differ from those used in PRISM supervision. As 
planned, however, the district recently modified the DEBE-333 to include 
PRISM terminology. The new form will be used in the 1985-86 school year. 

The five-member staff development team is organized to carry out three 
basic roles, each of which occupy approximately one-third of their time. 
Specifically, they: (1) engage in follow-up coaching of administrators; 

(2) teach PRISM content and conduct observation clinics at SHTC; and 

(3) are responsible for separate coordinator roles, each of which involve 
planning, liaison and materials development activities. The project 
director coordinates the administration of the SDT. The other staff 
coordinate activities related to research and evaluation, system-wide 
implementation/follow-up, inservice training, and administrative growth and 
development , respectively. 

Monitoring /Evaluation 

The district has employed an extensive monitoring system to track the 
implementation of PRISM. Principals have been required to submit monthly 
reports to SDT staff on the number, type and focus of observations/confer- 
ences conducted* They have also been required to submit mid-, and end-of- 
year progress reports on PRISM staff development plans/activities carried 
out in their schools. They also describe their PRISM activities in their 
annual MBO report to the deputy superintendent. Members of the SDT have 
also maintained confidential staff development coaching logs to document 
their coaching activities with principals. 

From the inception of the PRISM program, the district incorporated 
systematic evaluation procedures to monitor, adjust, and evaluate the 
effectiveness of all aspects of PRISM. Evaluation work has been carried 
out primarily by the district's Office of Testing and Evaluation with 
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assistance from the Learning Research and Development Center in Pittsburgh. 
Comprehensive evaluation activities have focused on: (1) program documen- 
tation (e.g., action plans, field notes, materials); (2) formative evalua- 
tion (e.g., post-training surveys); and (3) impact studies (e.g., assessment 
of attitude and teaching behavior changes). Evaluation results to date 
have shown positive results or changes developing in the intended direction. 

Project Costs 

The PRISM program has been included as a line item in the district 
budget since its inception in 1981. Approximate costs since that year 
through 1985 have been $130,000, $352,000, $684,000, $508,000, and $532,000, 
respectively. These monies have gone primarily to staff development team 
salaries, and teacher inservice salaries (i.e., salaries for staff associated 
with the Schenley High Teacher Center [SHTC] during training). Additional 
monies from the Ford Foundation ($445,000 over five years) and local 
corporations/foundations ($1,000,000) have been used to provide supplemental 
support for ancillary aspects of the SHTC program. 

Administra t ors 1 Perceptions 

Without exception, principals praised the utility and quality of their 
PRISM training. They generally reported increased confidence in supervising 
teachers to improve instruction and were receptive to additional SDT 
training in new areas. Perceived program strengths, in addition to quality 
training, were the common language of instruction, strong/sustained district 
commitment, SDT coaching, slow introduction of the program, large number of 
observations, positive reinforcement of teachers, and the teacher center. 

All of the principals felt that PRISM inservices and related super- 
visory observations have had a positive impact on teachers 1 instructional 
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behaviors. They also perceived improvement in their rapport with teachers 
and increased support for teachers. Although they still saw evaluation as 
somewhat of a pro forma exercise, several principals noted that this was 
changing due to the superintendent's modeling of personnel evaluation 
standards. Major problems still to be resolved were finding time for 
observations, and the amount of paperwork required. Some also mentioned 
that the district was approaching the overkill noint regarding PRISM 
training. 

Teachers T Perceptions 

Most teachers viewed PRISM as a definite improvement over the DEBE-333 
observations conducted in the past. They perceived that since a common 
language and expectations had been established, the observations and annual 
evaluation were more objective. They also cited increased communications 
between teachers and administrators regarding instructional improvement as 
a plus. Nearly all teachers reported t/at PRISM had helped their teaching 
performance. At least half named specific changes in their teaching 
behavior. Some noted, however, that some principals had introduced PRISM 
more thoroughly and effectively than others. 

Most teachers said they were observed 4-6 times during 1984-35. When 
asked about the percentage of their teaching knowledge and skills that they 
would attribute to PRISM inservice/supervisory experience, they responded 
as follows: forty percent of the teachers attributed 10-15 percent of 
their teaching knowledge and skills to PRISM, 20 percent attributed 20 
percent to PRISM, and 40 percent attributed 25-35 percent to PRISM. 
Regarding evaluation, most viewed the DEBE-333 as a pro forma exercise. 
Perceived problems included variability in how thoroughly different 
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administrators had introduced PRISM and differences among teachers in the 
frequency and scheduling of observations. 

Central Office Staff Perceptions 

Central office staff cited many program strengths. Among the most 
salient were: (1) the development of the SDT; (2) use of an R & D-based 
staff development model which employed extensive coaching; (3) use of 
principals to train teachers; (4) numerous observations; (5) omission of 
"official observation write-ups"; (6) use of positive reinforcement in 
teacher conferences during the first two years to build trust; and (7) the 
formal connection of PRISM and DEBE-333 four years after program introduc- 
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Major problems, according to central office staff, were in training 
the SDT, developing materials, using principals to train staff, the time 
factor regarding principals' observations, and various stresses at the SHTC 
regarding the roles of PRISM support staff that had resulted in some 
teacher burnout. 

Based on their experience, Pittsburgh central office staff would 
advise other districts planning a TS/E program to have a clear vision of 
desired TS/E goals and procedures, involve teachers in planning and use 
their input, choose a single instructional model, train all staff in depth, 
provide for much initial positive reinforcement and follow-up coaching, and 
be flexible. Staff also noted that the Hunter model is not the 
end-all-be-all of instruction. They consider it a common foundation or 
frame of reference for effective instruction. It should be used to build 
staff skills toward more subject-matter specific instructional skills. 
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Overall Program Significance/Implications 

In many ways the TS/E improvement efforts in Pittsburgh Public Schools 
over the past four years speak for themselves. Overall, the improvement 
effort can be characterized as being well planned, designed, implemented 
and evaluated. The district's efforts are noteworthy for their clarity of 
purpose, comprehensive nature, use of intense and sustained staff develop- 
ment, R&D-base, use of systematic monitoring and follow-up procedures, and 
even for the superintendent's modeling behavior. Many implications 
regarding TS/E program planning/ startup , design, training, and implementa- 
tion/continuation can be drawn from Pittsburgh's experiences. Three are 
particularly significant. First, Pittsburgh's experience appears to offer 
a compromise approach to a fundamental TS/E dilemma. Namely, how to 
present a program to teachers as "improvement oriented" and then "tie it to 
evalvation. " Pittsburgh's solution was to incorporate a four-year phase-in 
pericd to build staff skills and trust before making the forma?, connection 
between their supervisory (helping) and evaluative (rating) procedures. 
Secord, Pittsburgh's decision to involve principals in training staff 
initially resulted in resistance. It appears, however* to have paid off in 
the end in increased administrator supervisory skills and confidence. 
Finally, changes in staff behavior (principals and teachers) are not going 
to occur overnight. Sustained, long-term staff development, follow-up 
coaching, positive support and reinforcement, and systematic monitoring and 
follow-through are required if real improvements are to occur. 
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Summary of Case Study 4: Tamaqua Area School District 

Tamaque Area School District has five schools, three elementary, one 
junior high, and one high school. There are 7 administrators, 140 
teachers, and 2,450 students. The district has an evolving observation 
system (with no official label) and uses the DEBE-333 for end-of-year 
evaluations . 

TS/E Program History and Philosophy 

The TS/E system was initiated by the current superintendent when he 
assumed that position in 1978-79. Before that, the district had no formal 
supervision/evaluation procedure. The superintendent identified seven 
components of a successful lesson (e.g., providing for readiness, stating 
objectives, and evaluating student learning), and used them as observation 
criteria in a modified clinical supervision format. 

District administrators view the observation system as a more useful 
form of evaluation than the DEBE-333 because of its management and super- 
vision features. In their opinion, it helps them establish expectations, 
acknowledge good teaching, assist individual teachers, and identify common 
staff development needs. 

Staff Development 

Training for administrators and teachers has not been extensive or 
formalized. Administrators have become knowledgeable about the system 
through tutoring by the superintendent, discussing it during administrative 
staff meetings, and attending relevant workshops (e.g., sessions conducted 
by PDE and IU 29). Teachers have learned about the system through reviews 
of their observation reports, presentations by principals at faculty 
meetings, and descriptions of observation criteria. 
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TS/E Organization 

Generally, elementary teachers are observed every two years and 
secondary teachers every year. The supervision process includes observing 
a classroom during an entire period; preparing written reports that include 
a timed record of events, a statement about teacher preparation, an evalua- 
tion of the lesson observed, and recommendations; and conducting post 
conferences. The district does not have an official, standardized set of 
observation criteria; each observer uses a variation of the successful 
lesson components list developed by the superintendent. 

Supervision/evaluation in the three elementary schools is primarily 
the responsibility of the principal, although the superintendent and 
assistant superintendent observe some teachers. Junior high and high 
school TS/E responsibilities are organized by department. The superinten- 
dent, assistant superintendent, secondary principal, and assistant princi- 
pals are each assigned to one or more departments. (They also serve as 
chairpersons of those departments.) Each secondary school administrator is 
expected to conduct a minimum of 10 observations per year, Minimums have 
not been established for other administrators, but they reported conducting 
15-20 per year. 

Monitoring/Evaluation 

The system has not been evaluated formally, but the superintendent 
monitors it closely. He receives copies of all observation reports, 
reviews them, and then confers with individual administrators. He also 
maintains records showing the dates when each teacher was observed and the 
number of observations conducted by each administrator. 
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Program Costs 

The TS/E system requires little additional funding and is supported 
out of the district's budget. 

Administrators' Perceptions 

Administrators considered the opportunity to increase communication 
with teachers and to work with them to improve instruction to be the 
system's major strength. Most indicated that the observation system has 
helped them improve teaching performance, and some said that it has helped 
them relay their expectations to teachers or has prompted curricular 
changes such as discontinuing a reading series. Other positive outcomes 
are increases in administrators 1 knowledge of classrooms and the 
development of observation criteria and guidelines. Several administrators 
noted, though, that teachers' reactions to the system were generally 
negative. For example, teachers resented the time spent on conferencing, 
perceived recommendations as negative evaluation statements, and looked 
upon the criteria as nothing new. Also, all five administrators expressed 
a need for additional training, primarily in conferencing skills, in 
helping teachers overcome weaknesses, and in using the observation 
criteria. Other problems included lack of time for observation, inadequate 
training for teachers, and too few observations and pre-conferences. 

Teachers ' Perceptions 

Teachers' exposure to the observation criteria varied, depending on 
their principal. Some received lists of the criteria, without definitions; 
some heard oral presentations of them; some never saw or received a complete 
list of them. Some learned of the criteria through conferences, deduced 
them from observation reports, or learned about them from other teachers. 
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Some said that the system has helped slightly or has the potential to 
help--by allowing administrators to give teachers constructive criticism 
and by keeping teachers on their toes. Also, they liked the fact that it 
increased administrators' knowledge of their teaching. 

The strength that teachers mentioned most frequently was the oppor- 
tunity for communication with administrators during conferences. Other 
strengths included the system's fairness, its comprehensiveness, and its 
non-threatening nature. Most teachers, however, said that the observation 
system did not help improve their teaching performance. They said that 
observations failed to identify weaknesses, were not taken seriously 
because reports would include recommendations even if the lesson observed 
was perfect, were too infrequent, held them accountable for criteria that 
had not been communicated to them, and were conducted by administrators who 
were not trained in their subject area or who did not know them on an 
informal, daily basis. 

Central Office Staff Perceptions 

District administrators said that the system's strengths included the 
amount of time required for each observation, the needs assessment utility 
of the information produced, and the improvement of instruction— although 
the impact on the latter might be greater if observations were more 
frequent and more teachers accepted the system. Problems have included 
administrators' tendency to remain in their offices rather than conduct 
enough observations, lack of time for observation, deficiencies in 
administrators' skills in helping teachers overcome weaknesses, and 

-.hers' lack of knowledge and acceptance of the system. Central office 
advised districts who adopt new TS/E systems to obtain input from 
tors 



during system development and to make deliberate efforts to sell the system 
to staff. 

Overall Program Significance/Implications 

With little external assistance or expenditure of funds, Tamaqua 
adopted an alternative TS/E system that is similar to much more expensive 
systems used in other Pennsylvania districts. Among the elements found in 
the Tamaqua system that are also in these other systems are: (1) classroom 
observations by administrators who make anecdotal records; (2) criteria 
based on nationally-known research/theoretical knowledge on the process of 
instruction; (3) written reports rather than checklists; and (4) post- 
observation conferences between administrators and teachers. 

While Tamaqua has demonstrated that alternative TS/E systems can be 
introduced at relatively low cost, it has also illuminated some potential 
limitations that should be addressed. If training for teachers, usually a 
major expense, is restricted, districts need to find other ways to give 
teachers as much information as is possible about the system. Alternatives 
to intense training include distributing printed materials, holding 
inservice sessions, making presentations or. conducting discussions during 
faculty meetings, giving credit for participating in relevant courses or 
workshops, and tutoring during pre- and post-observation conferences. 
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Summary of Case Study 5: Upper Perkiomen School District 

The Upper Perkiomen School District has three elementary, one middle, 
and one high school. It is administered by four central office staff 
members, five principals, and two assistant principals. The district has 
approximately 165 teachers and 3,000 students . Ninety-eight percent of the 
students are white, and enrollment is stable. The district has a staff 
development program (Utility-Based Professional Staff Development-UPSD) 
that is based on the work of Madeline Hunter and others. Staff select from 
five alternative modes of S/E; one alternative employs only the DEBE-333, 
which is also completed annually for all staff, 

TS/E Program History and Philosophy 

The program was initiated by the superintendent in 1977, During the 
process of adoption, staff and others were involved extensively—in trips 
to the Newport News, VA school district, in a pilot workshop, in inservice 
sessions, and in presentations by administrators. The subsequent develop- 
mental process included additional trips by the superintendent and the TS/E 
trainer to both coasts to consult with experts (Hunter and Stachowski) and 
observe the Newport News program, visits to Upper Perkiomen by Newport News 
staff members, and consultation of literature on TS/E. The overall district 
philosophy is that teachers' instructional skills are critical and that all 
teachers are satisfactory. However, administrators also believe that the 
TS/E program can help improve the skills of even above-average teachers. 

Staff Development 

Administrators received extensive training on the content of the 
Hunter instructional model and various clinical supervision procedures. 
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The training process included trips to Newport News, a one-to-two week 
training session conducted in Upper Perkiomen by a Newport News administra- 
tor, a four-day workshop led by the director of Pittsburgh's TS/E program, 
and meetings with various consultants. 

Teachers were trained in the instructional model. Training topics 
included teaching to an objective, set, closure, task analysis, and moni- 
toring the learner and making adjustments. Training was conducted by a 
former high school reading specialist who, upon the recommendation of a 
committee of teachers, was given a five-year assignment as program trainer. 
Training lasted 6-8 weeks and consisted of cycles of one day of group 
instruction followed by four days of practice and observation while teaching. 

All administrators and 80 percent of the teachers were trained during 
the program's first four years. Administrators were trained first and then 
20-40 teachers received training each year. 

Requirements and Organization 

Teachers have the option, each year, of choosing one of five super- 
visory modes, subject to the approval of their building administrators. 
Four of the five modes vary according to their focus and who conducts the 
clinical supervision. Teachers have the following options. They may 
engage in a clinical supervision process conducted by the principal; or 
engage in a peer-clinical observation process with one or two of their 
peers — to focus on select aspects of general instructional improvement. 
Alternatively, they may engage in a clinical observation process with 
either the reading supervisor (K-4) or the language arts supervisor (5-12); 
or engage in a clinical observation process with the writing program 
specialist — to increase instructional skills in those specific content 
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areas. Lastly, they may engage in a supervisory process that is strictly 
evaluative and which employs the DEBE-333 as the basic observation/reporting 
instrument. All non-tenured and new teachers are assigned to the latter 
mode. 

Teachers' selection of alternative modes helps determine the distribu- 
tion of responsibilities for conducting the TS/E process. Principals are 
responsible for teachers who select two modes — the one in which the 
principal is the supervisor, and the evaluative mode. The reading super- 
visor and the writing program specialist are each responsible for one mode. 
The district supervisor of curriculum, instruction, and personnel observes 
non-tenured teachers and helps oversee the collegial supervision mode. The 
superintendent and assistant superintendent manage the process and help 
conduct observations. 

The number of observations required varies according to mode, but is 
not less than one per year. The supervision process is flexible: pre- 
conferences are not always held; formal reports may be prepared before or 
after post-conferences; several alternative report formats are used, 
including a checklist; and principals may decide to delay reports and 
popt-conferences until after several observations. Only the reports of 
non-tenured teachers are sent to the district offices. 

Monitoring /Evaluation 

District administrators have used several approaches to monitor the 
process: requiring principals to submit all reports to the district 
offices (used initially only); participating in principals 1 and teachers' 
(collegial mode) observations and conferences; videotaping observations and 
conferences; and including TS/E tasks in administrators' accountabilities. 
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The program was formally evaluated in 1981 by researchers from 
Pennsylvania State University* They found that staff were very supportive 
of the program and said that it gave them an increased sense of competence 
and professionalism. The researchers raised questions about the program's 
integration into the district's ongoing practices and about resistance of 
some staff to it. In addition to this evaluation, at the end of each 
training cycle the staff development trainer surveyed teachers 1 acquisition 
of knowledge and skills, along with their reactions to training. Generally, 
teachers reported that they had acquired the knowledge/skills necessary to 
the program and reacted positively to the training experience. 

Program Costs 

The TS/E program cost the district approximately $10,000 per year 
during adoption and conceptualization and $30 ,000-$40 ,000 per year during 
training. Sources of funding included a federal grant (Title IV-C) , the 
school board, and staff development funds from PDE. The largest expense 
was the salaries of teachers during training. 

Administrators 1 Perceptions 

All administrators said that they thought the program plays an impor- 
tant role in helping teachers improve their performance. Among its 
strengths, for example, are that the existence of alternative TS/E modes 
encourages more people to become involved; the flexibility of reporting 
requirements enables administrators to convince teachers that the program's 
purpose is to supervise rather than evaluate; and the changes in the 
administrative role make it less threatening and more he/.pful. The program 
has improved communication among teachers and between teachers and adminis- 
trators. Among other effects, it has helped focus teacher discussion more 
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on instruction, established expectations, and improved lesson planning. 
Administrators considered their program training extensive and effective, 
raising questions only about a few consultants whose sophistication in TS/E 
was less than the districts. All but one expressed a need for further 
training in, for example, conferencing, note taking, peer observation, 
dealing with experienced teachers > and additional elements of Hunter's 
description of teaching. 

Teachers ' Perceptions 

Teachers mentioned several program design strengths — among them, the 
availability of alternative TS/E modes, particularly the collegial mode* 
the opportunity for practice and immediate feedback during training; and 
other features that reduce the program 1 s threat. Other strengths named by 
teachers included the staff development trainer; administrators' assurances 
that the program was not intended to reduce their individuality; and the 
district's allocation of time lor the process. 

Many teachers reported that the program had an impact on their 
teaching, but many others tried to minimize that impact. The effects named 
by teachers included increased understanding of how and why their own 
teaching behaviors affect student learning; improved planning and ques- 
tioning behaviors; a common vocabulary; increased awareness; and a new 
terminology for old behaviors. Some viewed the latter as useful, saying it 
helped keep them on their toes; others questioned whether it justified the 
time and money spent on the program. Several teachers reported impacts 
from the collegial mode of supervision, including benefits derived from 
observing other teachers as well as from having others observe them. 




The major need that teachers expressed was for follow-up and review. 
Teachers also described two problems— the time required for training, and 
inconsistency between administrators' initial messages (that the program 
was voluntary and non-evaluative) and later behaviors (persuading teachers 
to participate, mandating the program for new teachers, and using program 
terminology on DEBE-333 reports). 

Few teachers commented explicitly on the adequacy or quality of the 
training itself. However, some who described the program's impact referred 
more to training than continuing TS/E experiences. Many found the training 
useful; others considered it redundant with their prior knowledge. Many 
teachers expressed appreciation of the trainer, saying that his coaching 
was helpful and non- threatening . Some teachers also said that the training 
required too much out-of-class time and that the district has provided 
relatively little foDow-up or refresher training. 

Central Office Staff Perceptions 

District administrators seemed pleased with the TS/E system's design, 
but when they talked about, strengths they referred primarily to impact- 
such as establisbing common language and expectations, increasing staff 
knowledge and ability to recognize teaching effectiveness, and improving 
the effectiveness of principals as supervisors and of teachers as instruc- 
tors. One administrator did describe the strengths of extensive teacher 
involvement and of the availability of alternative TS/E modes. 

The problems that administrators described included creating readiness 
for the program (for example, building trust with teachers and convincing 
principals of the program's value); obtaining funds for training; finding 
time; developing teachers' skills beyond a rote level; and maintaining the 
system. 
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District administrators advised others in their position to make a 
commitment to the program and get the board to do the same, perhaps including 

signing a 5-year employment contract for the superintendent; create readi- 

> 

ness for the program; understand the program concepts thoroughly before 

training teachers; and give teachers sufficient training* 

Overall Program Significance/Implications 

Upper Perkiomen's TS/E system was carefully planned and implemented, 
with many opportunities for staff to participate in the adoption decision, 
visit similar programs, consult with many scholars and educators, and 
extensively train for the program. The system is flexible and provides 
many options for teachers and administrators — for example, selecting among 
alternative modes of supervision and flexibility in preparing observation 
reports. It was also significant that a capable and trusted insider was 
the primary trainer of teachers. 

Upper Perkiomen f s TS/E system provides many guidelines for other 
districts that decide to adopt their own TS/E systems. Although most of 
ijts experiences were positive, these were a few instances of what districts 
should avoid in implementing TS/E systems — for example, telling teachers 
that participation is voluntary and later persuading them to participate or 
mandating the program for new teachers; telling teachers that a program is 
non-evaluative and later using program terminology on evaluation reports; 
and not providing sufficient follow up or refresher training. 
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Findings and Recommendations 

This study of teacher supervision/evaluation systems was limited to 
only five districts and did not examine program effectiveness from an 
outcome or summative evaluation perspective. Nonetheless, it did produce 
extensive descriptive and perceptual information for administrators to use 
in adopting new TS/E programs. Moreover, it enabled researchers to identify 
key components or characteristics of the TS/E adoption and implementation 
process. The presence or absence of those characteristics frequently 
helped explain why successes or problems occurred. Many are compatible 
with the findings of other researchers (e.g., Darling-Hammond, et al., 
1984) or the advice offered by staff development experts (e.g., Joyce and 
Showers, 1984). 

This section describes some critical elements in designing and 
implementing new TS/E systems. Then it discusses several key issues in 
greater depth. Following that, there is a description of several promising 
innovative practices used in the districts studied. Finally, it identifies 
external assistance needs. 

Critical Elements of TS/E Systems 

Critical elements of TS/E systems are discussed here according to four 
phases of program development/implementation. The four phases are: 
(1) introduction/generation of staff support, (2) system design, 
(3) training, and (4) implementation/continuation. 

Introduction/generation of staff support . TS/E systems are not likely 
to improve teaching unless teachers are willing to change their classroom 
behavior. Therefore, it is particularly important that teachers accept 
TS/E systems and consider them legitimate sources of guidance. Specific 



characteristics, when included in the introduction phase, can promote such 



acceptance. 

• Strong district commitment to a long-term improvement effort . 
Commitment in the form of strong leadership, repeated public 
endorsement of the system, and allocation of substantial 
resources—particularity staff time—not only spurs development 
and implementation but also conveys the sense that the district 
considers TS/E an important, long-term priority. Commitment also 
involves providing sufficient financial resources for consultants, 
visits to exemplary TS/E programs, materials, etc. Once a 
commitment was made, all the districts studied, even the smaller 
ones, found ways to allocate monies for planning and staff 
development. 

• Extensive planning phase . Adopting a new TS/E system is a major 
undertaking and requires substantial planning and preparation. 
Consequently, an extensive planning phase, with data-based program 
planning, is another characteristic that generates staff support* 
The planning process may include examining exemplary programs, 
hiring credible consultants (these are often administrators of the 
exemplary programs) , and/or conducting limited pilot tests of 
TS/E-related staff development. Consultants are often rich sources 
of information and advice on program design. They may also help 
legitimate the effort and motivate staff. 

• Adequate representation . All parties— teachers , union repre- 
sentatives, principals, other administrators, and perhaps community 
members — should be represented in the planning phase. 
Participation may relate to either establishing district TS/E 
goals, and/or operationally defining them. The important thing is 
that participation be genuine. A negative effect can occur if 
people believe that decisions have already been made or that their 
opinions carry little weight. 

• Clarification of the relationship between training/supervision 
and evaluation . The relationship between supervision and annual 
teacher evaluations must be clearly communicated to everyone early 
in the process. Subsequent procedures must then be consistent with 
that intent. Teacher support and trust is jeopardized when 
administrators initially state that supervision and evaluation will 
be separate activities but later allow the former to influence the 
latter. 

• Adequately portraying the proposed program's impact on teacher 
behavior . Similarly, administrators should avoid trying to make 
the system more tolerable to teachers by claiming that it simply 
requires their putting new labels on what they already do. This 
establishes an expectation that training/observation will not 
influence teacher behavior and that the district's adoption of 
the system was politically motivated or an otherwise empty 
formality and a waste of resources. 
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• Clarification of the status of teacher participation . District 
expectations regarding whether all staff will eventually be 
required to participate in the program should be clearly 
communicated early in program introduction. Staff trust in 
administration declines if they are told that a system is 
voluntary but are later pressured to participate. 

• Emphasis on the long-term/ongoing nature of the program . School 
staff are all too often subjected to quick-fixes and cosmetic staff 
development. As a result, "this too shall pass attitude" prevails 
in many districts. One site administrator reported regret that the 
program had initially been sold to staff as a three-year project 
because some staff were willing to wait three years for it to 
terminate. He recommended that TS/E programs be sold as ongoing 
and developmental. 

System design . Districts will need to develop comprehensive, written 

plans that describe the major elements of the TS/E system. This 

documentation helps guide system users, standardize implementation 

procedures across schools, and convince teachers that the district is 

attempting to implement the program systematically. 

• Specific model of teaching and observation criteria . A model of 
teaching — and, from that, observation criteria — should be selected 
and then described clearly so that all staff (administrators and 
teachers) have thorough, shared understandings. The model and the 
criteria should be derived from research and/or applied knowledge 
about effective teaching. Ideally, they should be appropriate for 
or adaptable to a variety of classroom situations. The strengths 
and the limitations of the model chosen should also be clearly 
communicated to staff. No one model is a panacea. 

• Suggested observation schedule . The program design should also 
recommend a schedule of observations. Generally, teachers should 
be observed at least twice each year, and more frequently if 
possible, especially during the first year or two that they use 
the model. This helps the observer understand a particular 
classroom situation, increases his or her credibility with 
teachers, and provides teachers with more coaching and feedback. 
However, observers should have the flexibility, as the program 
matures, to focus their attention on teachers who need assis- 
tance. This may mean spending less time with teachers who have 
long-term records of excellence. These latter teachers may be 
used, however, as models for peer observation or to assist with 
staff development. 
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Mo dified clinical supervision format . Districts should consider 
adopting a modified clinical supervision model. In its ideal form 
clinical supervision involves: (1) pre-conf erencing/goal setting, 
(2) classroom observation/data collection, (3) data analysis, and 
(4) post-conferencing (most districts studied did not use pre-con- 
ferencing). It may also involve beginning-of-the-year goal setting 
and should also include follow-up observation cycles where needed. 
The district model should involve script- taping (verbatim notes) 
as the primary mode of classroom observation data collection, and 
should also provide for different types/levels of conferences 
(e.g., Hunter, 1980). Conferences can vary in terms of: (1) 
whether the focus is on either positive or negative findings, or 
both; (2) who identifies the findings, the teacher or the observer, 
or both parties; and (3) who suggests solutions for identified 
problems and/or desired areas of teacher growth/extension. Dis- 
tricts would also be advised to emphasize positive feedback confer- 
ences during staff training and for some time after that (one year 
or more) , to afford time for staff practice of the discrete compo- 
nents of the model. 

Structured but flexible reporting procedures . Districts should 
provide guidelines/formats that observers can use in writing up 
observation findings and communicating them to others. These 
guidelines should indicate whether observation reports are to be 
sent routinely to the district office or maintained in school 
files. Sending reports to the district office is definitely 
more threatening to teachers. Districts should strongly 
consider using informal reporting systems, particularly during 
training and for the first year or two after training until both 
teachers and observers have had sufficient time and practice to 
internalize their skills. Districts should also consider intro- 
ducing some flexibility into the system so that observers can 
use the system as needed— for example, to observe a classroom 
informally without writing a report, to write up only a partial 
report when following up on a previously identified weakness, or 
to observe a teacher several times before writing a report. 

Inc orporation of the observation criteria into th e end-of-year 
ev aluation " If a district decides that the model of teaching 
should be considered in end-of-year evaluations, it should adopt an 
evaluation form (a modified DEBE-333) that explicitly includes the 
observation criteria. 

Coordination of the TS/E system with the staff development 
program . Despite extensive initial training in a TS/E system, 
teachers need periodic follow-up and an opportunity to meet in 
support groups to discuss their application of the model. Also, 
districts may want to extend that training or to modify aspects 
of the teaching model. Staff development time provides an 
excellent opportunity for such activities. In addition, 
districts should avoid inservice sessions that appear incom- 
patible with the model of teaching. 
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Training , Whatever TS/E system is selected, districts will need 
to provide extensive training for administrators and teachers. Otherwise, 
they cannot be expected to carry out the system appropriately. Initial 
training will help teachers acquire the skills required of them. Subsequent 
observation and conferencing will help refine and maintain those teaching 
skills. Effective training programs have several characteristics in 
common • 

• Use of effective training strategies . Training programs should 
incorporate techniques that experts have identified as effective. 
For example, Joyce and Showers (1984) described providing theory, 
demonstration/modeling, guided practice and feedback, follow-up 
coaching, and periodic review/support as essential steps in 
training. These techniques are applicable to administrators 
learning to be observers/conf errers and to teachers learning to 
use the instructional model. 

• Multi-year phase-in period . Districts should allow one or more 
years from the beginning of teacher training until teachers are 
held formally accountable for following the teaching model. There 
are two reasons for this: (1) it is likely to take several years 
to train all teachers, and (2) after training, teachers need time 
and feedback to develop and refine their skills. During that 
period, observation reports should not be placed in teachers 1 
permanent files or sent to the district office— they are best kept 
in a non-permanent file by the principal. This significantly 
reduces teachers 1 fears that they will be prematurely held 
accountable. At one site in the study, observers turned their 
observation/conference notes over to teachers during training and 
post-training follow-up. 

• Development and use of local trainers . Using local personnel 
(particularly principals but also district staff and adminis- 
trators) for training strengthens a districts instructional 
leadership, and improves communication and relationships between 
staff and administrators. Also, an internal training capability 
assures that teachers who enter the district after the initial 
training period receive adequate training and that everyone 
receives follow-up review and assistance. 

• Use of volunteers to advocate the system . Some districts may 
find it helpful to train volunteers to become informed program 
advocates. These volunteers can promote the system among their 
colleagues. In addition, by making teachers program advocates 
administrators can avoid becoming directly embroiled in whatever 
internal tensions the new system might create. 
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• Cross-grade t raining programs . It may be appropriate, at times, 
to train teac hers from different grade levels in the same group. 
This helps increase communication and breaks down political units. 
On the other hand, teachers may be more receptive to training it 
they have the support of familiar colleagues and continuing contact 
with them. One solution might be to create cross-level training 
groups with two or three teachers from each of several elementary 
and secondary schools. 

• Formative e valuation of training . Formative evaluation can _ 
"provide information for improving a training program and identi- 
fying further training needs. To maximize its utility, each 
session should be evaluated and the results used. The credi- 
bility and effectiveness of the training program are enhanced 
when participants' suggestions are used to make improvements. 

Implementation/continuation . After the training phase has ended and 

the TS/E system has begun operation, district support will still be needed 

to make the system function effectively. There are several aspects to this 
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Time for observation and conferencing . After the enthusiasm 
and commitment that often accompany the early stages of a new 
program begin to wane, other priorities may take over and interfere 
with time originally set aside for observing and conferencing. To 
prevent this from happening, districts might consider assigning 
some of the principals' duties to others. Another possibility is 
time management training for administrators and instituting 
procedures for systematizing/delegating routine administrative 
duties . 

Follow- up training . Teachers are likely to need periodic 
follow-up training to refine and extend their skills, review the 
teaching model, and renew their enthusiasm. Although inservice 
time may be used for this, districts should ask themselves ^ 
whether that is sufficient, particularly during the program s 
initial years. Teachers could be given additional released time 
for improving their instructional skills. 

Support groups. Both administrators and teachers can benefit 
• from sustained support networks. These might meet two to four 
times a year and be organized by grade level, subjecc area, role 
group, etc. Support groups allow staff to share ideas and 
problems and can assist the district in monitoring implementa- 
tion, identifying needs, and making system adjustments. 

Sustai ned monitoring and program adjustment . Districts should 
monitor program Implementation and make adjustments whenever 
necessary. Monitoring could be as simple as keeping records of 
which teachers are observed when, and the kinds of conferences 



ERIC 



48 47 



that are conducted. Preferable, however, is a monitoring plan that 
includes procedures for assessing the quality of observations and 
conferences. Districts should be aware, though, that program 
monitoring has several potentially undesirable side effects—for 
example, reducing flexibility that helps increase the system s 
effectiveness and encouraging pro forma implementation— that should 
be considered when monitoring plans are designed. 

Key Issues 

People who design TS/E systems must make a variety of decisions. For 
example, they must select a model of teaching, decide how to train adminis- 
trators and teachers, identify observation criteria and a process for 
conducting and following through with observations, and' develop a strategy 
for enlisting staff support and cooperation. Generally, options exist but 
each with its own strengths and limitations. Moreover, decision makers 
have less than perfect knowledge of the likely consequences of various 
alternatives. The study particularly illuminated three issues which will 
be discussed in some depth here. They are (1) the relationship between 
supervision and evaluation, (2) the selection of the trainers, and (3) system 
monitoring. 

The relationship between supervision and evaluation . This issue is 
encountered by all districts that adopt new TS/E systems. On the one hand, 
common wisdom holds that teachers are more receptive to assistance that is 
non-threatening and, therefore, non-evaluative. On the other hand, observa- 
tions often produce data that are much more valid and useful for evaluation 
than some evaluation systems. Consequently, administrators may be tempted to 
use data collected for supervisory purposes evaluatively . A major cause of 
tension between teachers and administrators in the districts studied was the 
influence of the new TS/E system on teachers' annual evaluations. Although 
teachers were told that the new TS/E program was intended to help and not to 
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evaluate them, some still considered administrators 1 observations evaluative. 
Moreover, teachers took note of the fact that new TS/E terminology was used in 
their annual evaluations and that central office administrators were sent 
copies of their observation reports. Teachers' perceptions that administra- 
tors had deceived them probably created more tensions than if administrators 
had initially said that, after a reasonable phase-in period, observation 
criteria would be used in teacher evaluation. 

The best alternative may be to modify evaluation procedures to incor- 
porate the TS/E system's observation criteria. Before using the new obser- 
vation criteria evaluatively , however, districts should give teachers several 
years to acquire the necessary teaching skills. During that time, administra- 
tors should carefully avoid using program terminology in evaluation reports, 
or, if it is used, it should only be used to commend positive performance* 

In short, the district's position on supevision and evaluation should be 
that: (1) new teaching skills and expectations will become part of the 
supervisory teacher observation process; (2) an adequate training and phase-in 
period will be provided for both teachers and administrators, during which 
time teachers will not be held formally accountable for the new expectations; 
(3) during the phase-in period district and teacher union officials will 
negotiate how the new expectations will be related to teachers' annual 
evaluation; and (4) following this lengthly phase-in period, teachers will be 
held formally accountable for demonstrating the district's instructional 
expectations via their annual performance evaluation. 

The selection of trainers . One set of decisions facing TS/E program 
designers concerns who will conduct the training. The first question may be 
whether to use internal or external trainers. External trainers may have more 
expertise and credibility than local staff. However, they are likely to cost 
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more, be unfamiliar with the district and staff, and be available only at 
selected times and for limited periods. Internal trainers, on the other hand, 
may lack expertise but will probably cost less, understand how the district 
functions, and be more available. 

Even if a district opts for internal trainers, it must still decide who 
those trainers will be. Principals might be responsible for training teachers 
in their schools. Perhaps other personnel might be relieved of their duties, 
either on a full or part-time basis, to serve as trainers. Assigning training 
to principals is a low-cost option that has the additional advantages of 
helping develop their instructional leadership skills and improving their 
relationships with staff. However, training responsibilities substantially 
increase time demands on principals. And, as one district in the study 
learned, the extent and quality of training, as well as teachers' attitudes 
toward the program, are likely to vary across schools, depending on 
principals 1 knowledge, skills, commitment, and rapport with staff. In 
districts where this might be a problem, it may be preferable that training be 
conducted by a team that consists of the principal and a district trainer. 

Among the criteria used to select trainers are interest, the ability to 
acquire thorough knowledge of the teaching model and the observation criteria, 
teaching ability, and credibility and rapport with teachers. Since the 
trainer's role will be to work with teachers to improve their instructional 
skills, the latter attributes are particularly important. Trainers who lack 
experience or whose credibility is otherwise suspect may increase rather than 
decrease skeptics' resistance to the program. For that reason, districts 
should be wary of selecting vocal program advocates or ambitious personnel who 
have the right knowledge and incentives but lack credibility with staff. 
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Districts might also be advised to select two trainers, one with elementary 
level and the other with secondary level credentials. 

System monitoring . Monitoring may be conducted for several different 
purposes. Districts may want to verify that observations and conferences are 
conducted at the expected frequency. Or, they may want to assess the quality 
of those observations/conferences and provide feedback to administrators. 
They might also use monitoring to identify program adjustments or individual 
assistance needs. The districts studied employed several different kinds of 

monitoring activities: 

• Keeping r ecords of observations . At the most elementary level, this 
may simply involve recording which teachers are observed and when. At 
a more complex level, a district may decide to track not only who is 
observed when, but also the subject area observed, the type of 
post-conference employed, and the teaching model content areas most 
frequently discussed in the post-conference. Such records can be used 
to ascertain whether teachers are observed at the expected frequency, 
whether administrators are conducting enough observations, the subject 
areas observed, and the nature of feedback provided teachers. 

• Re viewing observation reports . District administrators may review all 
or selected observation reports to assess the quality of both the 
reports and the observations. 

• Sit ting in on observations and conferences . District administrators 
and/or trainers may sit in on observations and conferences between 
principals and teachers. This allows them to judge the quality of 
observations/conferences and provides opportunity for feedback to 
principals. Disadvantages to this are that it is time consuming and 
threatening to teachers. 

Estab lishing a minimum number of observations for principals . This 
can help ensure that the expected number of observations are actually 
conducted. However, establishing minimums can be tricky. Minimums 
can easily become maximums for principals in a time bind. If they are 
too high, they can lead to hasty and pro forma use of the system, or 
to other unintended effects such as principals neglecting other duties 
or not being availabile when needed. 

Videotapin g observations and conferences . Videotaping produces objec- 
tive, lasting records that can be used to assess the quality of obser- 
vations and conferences. Such records may be especially useful for 
tutoring both observers and observees. However, the procedure is ex- 
pensive, time consuming, intrusive, and may be very threatening to 
both teachers and principals. 
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• Requiring principals to submit periodic reports . Districts can ask 
principals to prepare annual or semi-annual reports describing their 
TS/E activities (e.g., observations conducted, assistance provided, 
training sessions held), assessment of the program, recommendations, 
and needs. This procedure can produce useful information if it is 
designed to be more than simple meaningless paperwork. 

When developing monitoring plans, districts should first identify the 

purposes that monitoring will serve. They should then consider alternative 

monitoring procedures and weigh the potental benefits and undesirable effects 

of each. One rule of thumb is to establish more systematic, detailed 

monitoring systems in the first few years of a program and then reduce 

monitoring as staff growth and program implementation is achieved. 



Promising Innovative Practices 

The TS/E systems in all of the districts studied included elements 
;hat merit duplication in the design of new programs. Two, however, stood out 
as particularly innovative. 

Alternative modes of supervision/evaluation . Upper Perkiomen allows 
teachers to select from different supervision/evaluation approaches. Super- 
visors and activities vary according to mode. Alternative modes are: 

• Principal-Teacher. The principal is the clinical supervisor. (In the 
high school, this responsibility is shared by two assistant princi- 
pals.) 

• Collegial Mode. Teachers observe each other. Participation in this 
mode requires administrative approval. Informal criteria for partici- 
pation are five years of experience and demonstrated competence. 

• Writing Program Specialist and Teacher. The writing specialist func- 
tions as the clinical supervisor. 

• Staff Development Coordinator-Teacher. The staff development coordi- 
nator serves as the clinical supervisor. 

• Evaluative Mode. This mode is viewed as "strictly evaluative." It 
employs as the basic instrument, the DEBE-333. Non-tenured teachers 
must use this mode. Others may choose it if they desire. 
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This existence of alternative supervision/evaluation modes has several 
advantages. First, it reduces principals' observation responsibilities (and, 
thus, time demands). Second, teachers who want to participate minimally in 
the new TS/E system have an option — the evaluative mode. (Districts could 
adapt the DEBE-333 to incorporate the TS/E observation criteria, thereby 
encouraging teachers to use the teaching model, but still require only annual 
observations and no post-observation conferences.) Third, the collegial mode 
increases teachers' sense of professionalism and their perceptions that 
administrators trust and respect them. In addition, this mode gives teachers 
an opportunity to visit other classrooms and obtain ideas that they themselves 
can use. 

Centers for teacher training . Pittsburgh Public Schools established the 
Schenley High School Teacher Center in September 1983. Quarterly cycles of 
approximately 48 teachers each have been scheduled for the program each year 
since the center opened. Teachers visit the center for eight-week periods to 
receive in-depth training in a clinical setting. During that time, their home 
school classes are taught by replacement teachers. The center, which is a 
fully functioning school in all respects, is staffed with teachers recruited 
because of their effective teaching abilities. Staff members, who teach a 
reduced class load and work with visiting teachers, have been trained in the 
district's staff development program. While at the center, teachers: 

• observe exemplary teaching in an ongoing school setting 

• practice new instructional techniques and skills 

• receive detailed feedback and analysis on that practice 

• apply instructional theory to practice 

• receive updates in the content of their subject areas 
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• review and discuss recent research on effective teaching, 
classrooms, and schools 

• study adolescent behavior and its implications for effective 
teaching 

• research chosen topics for personal growth. 

Teachers also engage in a follow-up program when they return to their home 
schools. 

The school district plans to open an elementary level teacher center 
in September 1985. Four weeks of intense clinical teaching experience and 
follow-up will be provided to each of the district's elementary school 
staff over the next four years. 

Establishing centers for teachers, to provide them with in-depth 
training and opportunity for renewal in a clinical setting is an extremely 
innovative concept. The centers in Pittsburgh have enabled the district to 
address both district goals (e.g., subject area content updates) and 
individual teacher improvement goals (e.g., practice of new instructional 
techniques and individual research topics). Evaluative data from the 
Schenley High School Teacher Center indicate that the program has 
contributed significantly to an increased sense of professionalism among 
secondary teachers. 

External Assistance Needs 

Central office staff from each of the sites studied indicated that 
they could have used and would have used assistance with TS/E program 
design and administrator training. Specific reference was made of the need 
for: (1) more well-developed and/or grade-level appropriate materials for 
training administrators and teachers; (2) video tapes of effective teaching; 
and (3) more state and/or intermediate unit leadership and assistance in 
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TS/E. Regarding the latter, one superintendent suggested that PDE provide 
skills academies for administrators in effective teaching and clinical 
supervision procedures. In effect, this would be a trainer of trainers 
program that would save districts the expense of hiring external consul- 
tants. 

All of the districts who did engage consultants were forced to seek 
out-of-state assistance. Trainers from these districts are now acting as 
TS/E design/training consultants for a number of other districts in the 
state, primarily during their less busy summer months. 

The findings of this study suggest that there is a definite and 
continuing support role for PDE in districts' TS/E improvement efforts. 
For example, PDE could: 

• continue to provide and consider increasing instructional improve- 
ment monies to districts in order to support TS/E efforts 

• disseminate select aspects of this study report to Pennsylvania 
school district officials to give them a perspective on a number of 
critical TS/E program initiation, planning, design/development, 
implementation and maintenance factors 

• serve as a clearinghouse for TS/E instructional models and staff 
development materials (e.g., training manuals, materials/handouts; 
observation and evaluation forms/formats; program monitoring and 
evaluation materials; and, videotapes of effective teaching) 

• bring in out-of-state national consultants and conduct centralized 
and/or regional trainer-of-trainers executive academies for PDE, 
intermediate unit, and/or local district staff 

• provide on-site design, training and implementation assistance 
through trained PDE staff, arrangements with PDE sponsored turnkey- 
trained intermediate unit staff, and/or arrangements with exemplary 
TS/E districts to free up existing in-state trainers to work with 
other districts in a more systematic fashion than is now occurring 

• make arrangements for several exemplary TS/E districts to serve as 
model demonstration sites (they do this to some extent already at 
their own expense — time and money). PDE might provide some 
financial support to these sites and to potential visitors to 
facilitate the process (e.g., staff time, materials duplication, 
travel costs) . 
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Some of the above alternatives obviously may be more feasible than others. 
Given the findings of this study and PDE's recent summary of the 1984-85 
Supervision/Evaluation Executive Academies, it appears that there is a 
broad market among Pennsylvania school districts for continued assistance 
with TS/E improvement efforts. Districts need to visit exemplary sites, 
have access to trained consultants, and have access to quality training for 
their own trainers. In addition, there is a continued need for state level 
financial support of districts' TS/E improvement efforts. 
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